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ABSTRACT 

We  consider  a  controlled  system 

X  =  f(x,u) 

exhibiting  a  self  excited  periodic  solution  x(t)  for  u(t)  =  0  and  consider 
the  question  of  modification  of  this  orbit  to  a  controlled  periodic  solution 
x(t)  corresponding  to  a  control  u(t)  chosen  so  as  to  minimize  a  cost 
functional  of  the  form 

A 

“  w(x(t),  u(t) )dt  , 

T 

A 

where  T  is  the  period  of  the  controlled  periodic  solution.  Some  relevant 
applications  are  cited. 
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SIGNIFICANCE  AND  EXPLANATION 


Great  pains  have  been  taken  to  make  Section  1  of  this  paper  an 
explanation  of  the  background  and  significance  of  the  work.  This  is 
particularly  true  of  the  first  three  pages  of  the  section.  This  material 
requires  minimal  technical  background  and  is  recommended  to  the  reader  wishing 
to  appreciate  the  basic  ideas  of  the  paper  without  getting  too  involved  in  the 
technical  material. 


The  responsibility  for  the  wording  and  views  expressed  in  this  descriptive 
summary  lies  with  MRC,  and  not  with  the  author  of  this  report. 
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OPTIMAL  ORBITAL  REGULATION  IN  DYNAMICAL  SYSTEMS 
SUBJECT  TO  HOP?  BIFURCATION* ** 

** 

David  L.  Russell 

1.  Introduction.  Many  systems  involving  self-excited,  nonlinear  oscillations 
can  be  usefully  modelled  by  a  system  of  the  form 

x  =  A(6)x  +  B(6)  (J  +  d(8)u  ,  (1.1) 

y 

x  e  En,  u  e  Em  , 

y  +  Y(®*y)y  +  k(8)y  ■  c(8)  x  +  e(8)  u  ,  (1.2) 

y  scalar. 

The  first  system  (1.1)  in  which  A(0),  B(0),  D(8)  are  matrices  of  dimension 

n  *  n,  n  x  2,  n  *  ra,  respectively,  represents  a  linear  oscillator  of  some 

sort  with  a  degree  of  internal  damping,  i.e.,  the  eigenvalues  of  A( 9)  have 

negative  real  parts  for  all  values  of  the  parameter  8  under  consideration. 

The  scalar  second  order  equation  (1.2)  in  y  represents  a  nonlinear 

oscillator.  In  it  c(8),  e(8)  are  vectors  of  dimension  n,  ra,  respectively, 

and  *  denotes  transpose.  In  most  cases  the  uncoupled  equation 

y  +  Y(8,y)y  +  k( 8)y  -  0  (1.3) 

has  y  ■  y  ■  0  as  an  asymptotically  stable  critical  point  for  8  in  some 

range,  say  0  <  8  <  0Q,  that  critical  point  becoming  unstable  for  8  >  8^. 

•  • 

Typically  the  terra  Y(9,y)y  takes  the  form 

* 
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Y(0,y)y  -  YQy  -  Y^Qty  +  Y(9,y)y  , 

where  YQ  >  0,  Sy^C®)  >0,  6^0.  The  higher  order  term*  represented  by 

A  •  •  • 

Y(9,y)y  provide  strong  restoring  forces  when  y  is  large*  The  familiar 
consequence,  amply  documented  in  the  literature  (see,  e*g.,  [A],  [B] ,  [C] )  is 
that  for  0  >  ®0  (1.3)  has  a  stable  periodic  solution  expanding  rapidly  away 

from  the  origin  (0,0)  as  ®  increases  beyond  0Q. 

The  uncontrolled  coupled  system  ((1.1),  (1.2)  with  u  =  0)  typically 
involves  a  matrix  B(0)  which  is  rather  small,  so  that  the  matrix  describing 
the  linearization  at  the  origin, 


is  nearly  lower  block  triangular  and  is  a  stability  matrix  for  0  <  0^ 
becoming  unstable  for  0  >  01 ,  where  0^,  the  bifurcation  point,  is  near  the 
0Q  value  already  discussed  for  (1.3).  The  system  (1.1),  (1.2)  then  likewise 
exhibits  a  stable  periodic  solution  for  0  >  0^,  lying  close  to  the  y,  y 
plane  for  0  near  8^ 

One  of  the  simplest  examples  is  the  Hartlen-Curry  model  ( [D] )  for  aero- 
dynamically  induced  flutter  of  a  long  elastic  rod  of  circular  cross  section  as 
shown  schematically  in  Figure  1.1.  A  stream  of  fluid,  e.g.,  air,  flows  past 
the  rod  with  velocity  V.  For  a  certain  "flutter  speed",  Vg,  the  fluid  flow 
is  steady  for  V  <  VQ.  For  V  >  VQ  the  fluid  flow  is  no  longer  steady! 
alternating  vortices  form  in  the  wake  behind  the  rod,  producing  a  near' 
periodic  force  of  alternating  direction  with  a  frequency,  dependent  on  V, 
which  is  known  as  the  Strouhal  frequency.  The  Hartlen-Curry  model  uses 
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equations  of  the  form  (cf •  [E] ) 


•  2 
x  +  foe  +  Px  ■  aV  y 

y  +  (O  -  oV)y  +  ~  y3  +  ®V2y  ■  bx 

to  describe  the  self-excited  oscillations  of  the  coupled  system  for  V  >  Vg 
(VQ  is  slightly  greater  than  0/a).  Typically  f$,  3,  0,  a,  b  are  rather 
small  quantities. 


Figure  1.1  Oscillations  of  a  rod  with 
circular  cross  section. 

A  more  complicated  example,  for  which  we  will  not  write  out  the  equations 
in  detail,  is  of  considerable  interest  in  flight  dynamics.  It  is  rather 
similar  to  the  example  just  presented  for  the  rod  with  circular  cross  section, 
except  that  we  now  envision  a  wing  cross-section  immersed  in  the  flowing  fluid 
as  shown  in  Figure  1.2.  The  displacements  shown  in  the  figure  are,  of  course, 
exaggerated  by  comparison  with  actual  operating  levels  (one  hopes!). 


Figure  1.2 

Here  z  represents  the  vertical  displacement  of  the  wing  tip,  t  is  the 
torsional  displacement  of  the  wing  at  the  tip,  and,  again,  y  is  the  vortex 
strength  measured  at  the  trailing  edge  of  the  wing.  One  obtains  here  a  six 
dimensional  system 

"  •  •  • 

X  -  f <Z,*,T,T,y,y,u,V) 

*#  •  •  • 

T  -  g(Z,Z,T, T,y,y,u,V) 

'*  •  •  • 

y  -  h(z,Z,T,T,y,y,u,V) 

exhibiting  self  excited  oscillations  much  as  in  the  Hartlen-Curry  case  with 
added  features,  such  as  divergence  of  the  periodic  solution  to  infinity  at 
certain  parameter  values  for  V. 

The  initial  oscillations  arising  in  systems  of  this  sort  are,  at  least 
insofar  as  the  x  component  (cf.  (1.1),  (1.2))  is  concerned,  of  rather  small 
amplitude.  The  exception  which  causes  greatest  concern  occurs  when  the 
nonlinear  oscillator  equation  (1.3)  has  a  periodic  solution  whose  frequency  is 
close  to  one  of  the  natural  frequencies  of  vibration  of  the  linear  elastic 
system  modelled  by  x  -  a(9)x.  When  this  situation  obtains,  quite  dramatic 
Increases  in  amplitude  may  result  in  (1.1),  (1.2).  Figure  1.3  shows, 
qualitatively,  the  sort  of  results  that  one  obtains  with  a  Hartlen-Curry  type 
model. 
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Figure  1.3 

Among  other  things,  control  may  be  used  in  (1.1),  (1.2)  to  increase  the 
range  of  parameter  values  6  for  which  the  origin  is  asymptotically  stable, 
i.e.,  to  increase  0^  and/or  to  modify  the  nature  of  the  oscillations  which 
do  occur  for  0  >  8^  -  by  suppressing  the  amplitude  of  those  oscillations,  for 
example • 

The  traditional  mode  of  control  is  linear  feedback,  the  general  form  of 
which,  for  (1.1),  (1.2)  would  be 

u  ■  Kx  +  l(.)  ,  (1.5) 

y 

possibly  with  gains  scheduled  so  that  1C  ■  K(0),  L  •  L(0).  it  will  be 
recognised,  however,  that  direct  measurement  of  y,  y  will  often,  perhaps 
usually,  be  impractical.  Certainly  the  measurement  of  the  vortex  strength  at 
the  trailing  edge  of  a  wing  would  require  a  sensory  tour  da  force.  So,  in 
practice,  (1.5)  must  ordinarily  be  modified  to 
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u  ■  Kx  .  (1.6) 

Whan  one  investigates  how  (1.6)  affects  (1.4),  one  sees  that  the  result  is  to 
replace  that  matrix  by 


A(8)  +  D<0)  K 


c(9)*  +  e(0)*K 


B(  0) 


[-V"  -ro  ♦'v”] 


0 


(1.7) 


One  may  verify  that  if  T  satisfies  the  quadratic  matrix  equation 

AT  -  Tr  -  TCT  +  B  -  0  (1.8) 

then  (1.7)  is  similar  to 


A  -  TC 
C 


CT 


°)  • 
+  r  / 


(1.9) 


For  0  >  0Q  r  is  unstable  and  it  may  be  assumed  that  K  is  selected  so  that 
A  remains  stable.  Zt  is  then  classical  that 

AT  -  Tr  +  B  -  0  _  (1.10) 

has  a  unique  solution  T  and  an  easy  application  of  the  implicit  function 
theorem  shows  that  for  small  B  (1.8)  has  a  unique  solution  near  the  solution 
of  (1.10).  Moreover ,  as  B  tends  to  zero,  T  tends  to  sero.  Wow  our 
capability  to  influence  the  eigenvalues  of  CT  t  T  lies  entirely  in  the 
term  CT  appearing  in  (1.9).  If  B  is  quite  small/  large  changes  in  K  are 
required  to  produce  modest  changes  in  T.  Zt  follows  that  when  B  is  small, 
as  is  coranonly  the  case,  it  will  be  difficult  to  materially  affect  the 
eigenvalues  which  are  responsible  for  the  bifurcation  phenomena,  i.e.,  the 
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eigenvalue*  of  CT  +  T.  This  means  that  restricted  feedback  of  the  form  (1*6) 
is  not  likely  to  be  very  effective  in  extending  the  range  of  values  of  6  for 
which  the  origin  remains  asymptotically  stable. 

It  may  be  argued  that  feedback  of  the  form  (1.5)  may  be  very  nearly 
realized  through  the  use  of  a  state  estimator  ( [F] )  for  y,  y.  However,  it  is 
likely  that  the  estimation  will  be  very  difficult  in  practice  because,  when 
B  is  near  0,  y,  y  are  nearly  unobservable  via  measurements  on  x  and, 
moreover,  the  oscillations  of  (1.2)  necessarily  take  place  in  a  region  where 
the  nonlinear  terms  appearing  there  balance  the  linear  terms,  "energy  wise", 
and  hence  are  of  comparable  magnitude.  Then  one  is  trying  to  construct  a 
state  estimator  for  nearly  unobservable,  nonlinear  phenomenal  a  rather  heroic, 
not  to  say  Sisyphic,  task. 

It  follows  that  if  linear  feedback  is  to  be  used,  it  will  take  the  form 
(1.6)  and  the  main  benefit  will  be  realized  through  choosing  K  so  that  the 
term  B(u)  (.]  in  (1.1)  affects  x  minimally  in  some  appropriate  sense.  For 

y 

example,  the  eigenvalue  of  A(0)  +  D(9)K  corresponding  to  the  fundamental 
mode  of  vibration  can  be  moved  further  to  the  left  in  the  complex  plane  or  its 
imaginary  part  can  be  increased  so  that  resonance  occurs  at  higher 
frequencies,  thus,  in  effect,  pushing  the  "spike"  of  Figure  1.3  to  the  right. 
The  main  point,  insofar  as  our  current  discussion  is  concerned,  is  that  one 
must,  in  effect,  concede  that  oscillation  is  going  to  take  place  and  then  take 
steps  to  modify  or  suppress  the  manifestations  of  such  oscillations  in  the 
system  represented  by  (1.1). 

Having  seen  that  we  may  as  well  admit  from  the  first  that  a  self-excited 
periodic  motion  will  be  present  in  the  operating  system,  we  may  as  well  cite 
this  as  an  excuse  and  an  opportunity  to  deal  directly  with  such  solutions  as 
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we  attempt  regulation  of  the  ayetem  (1.1),  (1.2).  Our  purpose  in  this  paper 
is  to  discuss  the  optimal  regulation  of  periodic  solutions  of  systems  of  this 
type,  primarily  with  a  view  toward  amplitude  suppression. 

Zn  Sections  2,  3  we  study  a  general  system  (wherein  6  is  suppressed 
until  needed) 

x  *  f(x,u)  (1.11) 

for  x  6  0,  an  open  set  in  r”,  and  u  6  rf*.  Most  of  our  results  are 
actually  obtained  in  the  context  of  the  specialized  systems  of  the  form 

x  -  g(x)  +  H(x)u  . 

Our  objective  is  to  show  the  existence  of,  and  to  characterize,  in  terms  of 
necessary  conditions,  state  and  control  pairs  x,  u  such  that  x  is  periodic 
with  period  T  and  such  that  x,  S  minimize,  relative  to  an  appropriate 
family  of  state  and  control  pairs  x,  u  (x  with  period  T),  a  cost 
functional  of  the  general  form 

J(x,u,T)  ■  ^  w(x(t),  u(t))dt  .  (1.12) 

In  Section  2  we  begin  by  supposing  that  the  uncontrolled  system 

x  -  f ( x,0 ) 

has  a  periodic  solution  with  period  Tg  and  then  obtain  some  local  existence 
results  ensuring  the  existence  of  an  optimal  control,  at  least  in  an 
appropriate  local  sense.  Then  in  Section  3  we  indicate  the  form  of  the 
necessary  conditions,  some  computational  methods  and  describe  some 
computational  work  already  undertaken,  concluding  with  an  indication  of  work 
in  progress  and  envisioned  for  the  future. 


2.  Local  Existence  of  a  Solution  for  the  Periodic  Optimal  Control  Problem 


In  our  study  of  existence  questions  we  will  confine  attention  to  systems 
in  which  the  control  appears  linearly s 

x  ■  f(x,u)  ■  g(x)  +  H(x)u  ,  (2.1) 

wherein  it  is  assumed  that,  for  some  open  subset  0  C  R11 , 

n  nm 

g  :  0  +  R  ,  H  »  0  -*•  R 

are  defined  and  continuously  differentiable  throughout  0.  Here  I^m  denotes 
the  space  of  n  x  m  matrices  with  real  entries.  The  differentiability  of  f 
with  respect  to  both  x  and  u  for  x  e  0,  u  6  fP,  is  then  clear. 

Our  basic  assumption  is  that  the  "uncontrolled"  system 

x  -  f(x,0)  ■  g(x)  (2.2) 

has  a  solution  x(t)  which  is  periodic  with  least  positive  period  TQ. 

Beyond  that  we  are  concerned  with  the  variational  system  based  on  x(t), 
which  is 

£  -  G(x(t))£  +  H(x(t))u  ,  (2.3) 

G(x(t) )  = (x(t)>  (Jacobian  of  g  w.r.  to  x) . 

3x 

We  denote  by  ${t,s)  the  fundamental  matrix  solution  of 

"  G(x(t))*(t,s)  ,  (2.4) 

4(s,s)  *  I  (n  x  n  identity  matrix)  (2.5) 

and  abbreviate  $(t,0)  by  Kt).  Both  (2.3)  and  (2.4)  may  be  taken  to  be 
defined  for  all  real  t  if  we  extend  x(t)  in  the  obvious  way  by 
periodicity.  The  resulting  systems  are  periodically  dependent  on  t  with 
period  Tg.  The  period  transition  map  associated  with  the  periodic  solution 
x(t)  is  the  map  defined  by  the  matrix  $(Tg). 


Since  the  system  (2.2)  is  autonomous,  we  may  stipulate  any  point  on  the 
solution  x(t)  as  the  initial  point  Xq  ■  x(0).  The  assumption  that  x(t) 
is  a  non-trivial  periodic  solution  enables  one  to  see  readily  that 

P1  -  x(0)  -  g(xQ)  0  •  (2.6) 

It  is  well  known  that  p.|  is  an  eigenvector  of  $(Tq)  corresponding  to  the 
eigenvalue  A^  »  1  of  *(TQ),  This  is  an  immediate  consequence  of  the  fact 
that 

£(t)  s  x(t)  5  «(t)Pl 

• 

is  a  periodic  solution  of  £  ■  G(x(t))£.  We  assume  that  the  remaining  n  -  1 

eigenvalues  of  ®(TQ),  which  we  denote  by  are  all  different  from 

1.  An  important  special  case  arises  when 

I A± |  <  1,  i  -  2,3, ...,n  »  (2.7) 

in  which  case  the  periodic  solution  x(t)  of  (2.2)  is  locally  asymptotically 

stable.  The  condition  (2.7)  is  not  needed  in  this  section  but  becomes  very 

important  in  numerical  considerations  to  be  introduced  later. 

We  denote  by  (p1 }  the  one  dimensional  subspace  of  Rn  spanned  by  p^ 

and  by  P  the  (n-1 ) -dimensional  subspace  spanned  by  the  (possibly 

generalized)  eigenvectors  p2»  p3,...,pn  of  ♦(T^)  associated  with  the 

eigenvalues  A  ,  A  , ...,A  .  if  p  is  a  real  n  *  (n-1)  matrix  of  rank 
2  3  n 

n-1  whose  columns  are  p2,  p3,...,pn,  or  consist  of  real  and  imaginary 
parts  of  p2,  p3,...,pn  in  the  case  of  complex  A^,  A^,...,!^,  then  each 
peP  has  the  unique  representation 

p  -  pa,  a  e  Rn-1  . 

Let  q1  be  the  unique  vector  in  Rn  such  that 
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q,  P,  ti  (pt''*i)Ra>  ■  1  • 


q1  p  “  0,  pep  . 

* 

Then  q1  is  an  eigenvector  of  4(Tg)  corresponding  to  the  eigenvalue  1 . 

We  denote  by  {q^}  the  one  dimensional  subspace  of  R*1  spanned  by  q1  and 

by  Q  the  ( n- 1  )-dimensional  subspace  of  R11” 1  spanned  by  the  eigenvectors 

q2,  q3,...,qn  associated  with  the  eigenvalues  A^Xj,...,!^  of  *(TQ)  (if 

we  were  to  allow  *(T)  to  be  complex,  .. . , would  be  replaced  here 

by  \  , A_, . • • , X  ) .  Letting  Q  be  the  n  x  (n-1)  matrix  whose  columns  are 
2  j  n 

q2,  q3,»»*,qn  (or  real  and  imaginary  parts)  it  may  be  arranged  that 

Q**  “  • 


Our  requirement 


X^  +  1,  i  •  2,3, ...,n  , 

guarantees  that  with 

0(t)  -  Q*(«(t)  -  I  )P  ,  (2.8) 

n 

0(Tq )  is  a  nonsingular  (n-1)  x  (n-1)  matrix.  Evidently  we  have  the 
decompositions 

{?1 }  e  p  -  Rn  -  {qi }  +  q  . 

Having  taken  care  of  the  various  definitions  and  assumptions  needed,  we 
turn  now  to  consideration  of  periodic  solutions  of  the  controlled  system  (2.1) 
corresponding  to  controls 

u  e  i£to,V,  Ti  >  T0  ' 

or,  more  precisely,  to  restrictions  of  such  controls  as  explained  below.  The 
theorem  which  we  present  below  may  be  proved  much  more  simply  when  we  require 
that  u  6  C [0 ,T^ ]  by  invoking  the  implicit  function  theorem  and, 
additionally,  a  local  uniqueness  result  is  obtained  in  that  case.  Because  of 
the  method  of  proof  employed,  and  in  the  interests  of  more  colorful 
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mathematical  metaphor,  we  shall  call  this  the  "arrow  and  target"  theorem.  It 

will  turn  out  that  the  proof  rests  on  the  intermediate  value  theorem. 

Theorem  2.1.  For  arbitrarily  small  positive  t,  6,  T  satisfying 

t  <  T1  -  Tfl,  there  exists  e  ■  e( t, 5)  >  0  such  that  whenever 

lui  <  e  (2.9) 

i/tO/T.] 

in  i 


the  system  (2.1)  has  at  least  one  solution  w(t)  satisfying 

lw( t )  -  x(t)  I  <  S,  0  <  t  <  T1  ,  (2.10) 

and  such  that 


w(T)  -  w( 0 )  (2.11) 

for  at  least  one  value  of  T  with 

IT  -  TQ|  <  x  .  (2.12) 

Once  w,  T  corresponding  to  u  have  been  identified,  w,  u  may  be 
regarded  as  a  periodic  state,  control  pair  by  first  restricting  w  and  u 
to  [0,T]  and  then  defining 

w(t  +  kT)  -  w(t),  u(t  +  kT)  -  u(t),  k  -  ±1,  ±2,»».  .  (2.13) 

Proof  of  Theorem  2. 1 .  For  simplicity  we  begin  with  the  equation 

w(t)  -  g(w( t) )  +  H(r(t))u  (2.14) 

wherein  r(t)  is  a  given  continuous  n-vector  valued  function  defined  on 
[0,T]  and  satisfying 

»r(t)  -  x(t)  I  <  5,  t«  [0 ,T, ]  , 

n  i 


x(t)  being  the  periodic  solution  of  (2.1)  corresponding  to  u(t)  =  0  which 
we  have  discussed  above.  This  includes,  of  course,  the  special  case  wherein 
H(x(t))  =  H,  H  constant  n  x  ra  matrix  . 
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Once  we  have  proved  the  existence  of  a  periodic  pair  w,  u,  for  this  special 

case  we  will  be  able  to  describe  rather  easily  the  modifications  required  to 

prove  the  corresponding  result  for  (2.1). 

2 

Let  u  e  L  [0,T  ]  satisfy  (2.9)  with  e  yet  to  be  determined,  and  let 
in  1 

w(t)  be  the  solution  of  (2.1)  corresponding  to  the  control  u  and  the 
initial  state 

w(0 )  *  +  p,  p  e  P,  Ipl  <  p  ,  (2.15) 

0  Rn 

for  some  p  >  0  yet  to  be  determined.  It  is  an  easy  consequence  of  the 
Caratheodory  existence  and  regularity  theory  (see  [G],  e.g.)  that  we  can 
assume 

»w(t)  -  x(t)  I  <  6  (2.16) 

Rn 

A 

for  any  6  >  0  by  choosing  e  and  p  in  (2.9),  (2.15)  to  be  sufficiently 

small.  Our  intent  is  to  keep  u  fixed  and  show  that  we  can  find  p  as  in 

(2.15)  and  T  satisfying  (2.12)  such  that  (2.11)  holds. 

Let  y( t)  be  the  solution  of 

y(t)  -  g(y(t))  +  H(r(t))u(t)  ,  (2.17) 

y(0)  -  x0  .  (2.18) 

Clearly  y(t)  satisfies  (2.16),  i.e., 

Iy( t)  -  x(t)  I  <  $  (2.19) 

Rn 

if  e  is  sufficiently  small.  Then 

w(t)  -  y(t)  +  z(t)  (2.20) 

where 

z(t)  -  g(y(t)  +  z(t))  -  g(y(t))  ,  (2.21) 

z(0 )  -  p  .  (2.22) 

The  variational  equation  associated  with  the  solution  y(t)  is 


C(t)  -  G(y(t))C(t> 


(2.23) 


do 

where  G(y(t))  *  (y(t))  is  the  Jacobian  of  g  evaluated  along  the 

A 

solution  y#  If  we  let  *(t)  be  the  matrix  solution  of 

*<t)  -  G(y(t))*(t),  t  6  [O.T^  ,  (2.24) 

*(0)  -  I  (2.25) 


and  set 

®(t)  -  Q#($(t)  -  I)P  ,  (2.26) 

the  usual  regularity  arguments  show  that  9{t)  converges  uniformly  to  9(t), 

given  by(2.£),  as  lu*  _  and  hence  sup  Oy(t)  -  x(t)l  })  tends 

Lm[0,TtJ  teEO.T^  Rn 

to  zero.  Since  9(TQ )  has  been  seen  to  be  nonsingular.  It  follows  that 

A  A 

0(T)  Is  Invertible  with  0(T)  satisfying  some  bound 

le(T)"1!  <B  ,  (2.27) 

provided  (2.9)  and  (2.12)  hold  with  e  and  t  sufficiently  small. 

The  "target"  of  the  "arrow  and  target"  theorem  is  the  image,  z^.  of  the 
subspace  P  c  Rn  under  the  map 

Z(p,T)  -  z(p.T)  -  p  , 

where  e(p,t)  *  z(t)  Is  the  solution  of  (2.21)  corresponding  to  the  initial 
state  (2.22),  i.e.,  z(p,0)  -  z(0)  *  p.  We  can  obtain  a  parametrized 
representation  of  zT  by  writing 

z(p,T)  -  p  -  a^p^  +  p,  p  e  P  ,  (2.28) 

P1  *  x(0)  -  g(x(0))  as  noted  earlier  and  (since  p  6  P)  setting 

°2 

P  -  pa,  a  -  (•  )  e  r""1  ,  (2.29) 

a 

n 


with  P  the  n  *  (n-1)  matrix  described  earlier.  Then 

* 

-  q^ztp.T)  -  p)  , 
a  »  Q*(z(p,T)  -  p) 


(2.30) 

(2.31) 
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and  writing 


p  -  P0,  6  6  r”"1  , 

we  see  that  the  Jacobian  of  cl  with  respect  to  0  at  0*0  (i.e.,  p  *  0) 

A 

is  precisely  6(T)  as  given  by  (2.26).  From  this  and  the  implicit  function 
theorem  it  follows  that  can  be  represented,  for  p  near  0  and  T  near 

Tq,  as  corresponding  to  the  set  of  points  (2.30),  (2.31)  for  which 

a,  -  <?(a,T)  , 

with  *P  of  class  C *  near  3*0,  T*Tg.  In  fact  (2.30)  shows  that 


(0,TQ)  q1  gp  (z(p,Tg)  -  P )  I  0  "ag  ^  Q,. o 


q*(*(TQ)  -  Z)pfof  f1 


(2.32/ 


The  surface  ZT  will  thus  be  nearly  tangent  to  the  subspace  P  at  p  *  0 

for  small  lul,  i.e.,  for  y(t)  near  x(t),  and  T  near  TQ,  because 

* 

*(Tq)P  *  PA  for  an  appropriate  (n-1)  x  (n-1) A  and  q^P  *  0. 

The  "arrow"  of  our  theorem  is  the  vector  Xq  -  y(T).  The  "arrow"  hits 

the  "target"  just  in  case  there  is  a  vector  p  e  P  and  an  instant  T  such 

that 

xg  -  y(T)  -  y( 0 )  -  y(T)  »  z(p,T)  -  p 

-  z( p,T)  -  z(p,0)  -  z(T )  -  z(0 )  , 


for  then 

w{0)  -  y(0)  +  z(0)  ■  y(T)  +  z(T)  *  w(T)  .  (2.33) 

That  such  a  "hit"  takes  place  is  a  consequence  of  the  Intermediate  value 
theorem  for  continuous  functions  on  R1.  The  situation  in  hand  is  represented 
graphically  in  Figure  2.1  where  ♦  •  ♦  •  ♦  is  the  trajectory  of  Xg  -  x(T) 
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for  T  near  TQ,  >  >■  -»  is  the  trajectory  of  Xq  -  y(T),  the 

hyperplane  P  is  shown,  and  the  curves  -----  indicate  the  outline  of  a 
cylindrical  region 

lx  -  (xQ  -  x(T) )  I  <  d  ,  (2.34) 

for  d  <  6,  truncated  by  hyperplanea  parallel  to  P 

H*  «  {x  |  x  -  kp1  +  p,  pep,  p^t-  ylpjl2)  ■  • 


Figure  2. 1 

The  surface  ZT  depends  on  u  via  the  dependence  of  z  on  y  in 
equation  (2.21).  Indicating  this  dependence  by  Z^fu),  we  begin  by  taking 
d  small  enough  in  (2.34)  so  that  Z,p^(0)  bisects  the  cylinder  into  two 
regions.  This  must  be  the  case  for  d  small  because  the  tangent  vector  to 
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Xg  -  x(T)  at  T  -  Tg  la  while  the  tangent  hyperplane  to  Z^ ( 0 )  at 

the  origin  is  P.  Next,  L  and  t  are  restricted  so  that 

P*(xQ  -  x(T) )  >  2L,  Tg  -  2T  <  T  <  Tq  -  T  , 

p*(xQ  -  x(T) )  <  -2L,  Tg  +  T  <  T  <  Tg  +  2T  , 

Then  we  make  £,  and  hence  lul  ,  small  enough  so  that  xQ  -  y(T) 

I«„(0,Tj 

K L  1 

lies  in  the  cylinder  (2.34)  for  |T  -  Tg|  <  T  and 

p*(Xg  -  y(T)>  >  L,  Tg  -  2T  <  T  <  Tg  -  T  ,  (2.35) 

p*(Xg  -  y(T) )  <  -L,  Tg  +  T  <  T  <  Tg  +  2T  ,  (2.36) 

both  of  Which  are  possible  because  y(t)  converges  uniformly  to  x(t)  as 
lul  tends  to  sero.  Finally,  e  is  further  restricted,  if 

m^'V 

necessary , 

so  that  Zm(u)  bisects  the  cylinder  (2.34)  when  lul  <6.  Because 

z(0,T)  =  0,  Z(0,T)  =  0  and  the  surface  ZT(u)  always  passes  through  the 
3  ip 

origin.  Then  —  (p,T)  can  be  uniformly  bounded  for  p  small  and  T  near 
3o 

Tg,  using  (2.27),  (2.32),  and,  further  restricting  d  if  necessary,  the 
intersection  of  ZT(u)  with  the  cylinder  (2.34)  does  not  meet  the  ends,  H*, 
of  the  cylinder  for  *ul  _  <  £,  |T  -  T.|  <  T,  Put  another  way,  if 

P^(T)  denotes  the  connected  neighborhood  of  the  origin  in  P  which  maps  into 
the  cylinder  (2.34)  under  Z(*,T), 

lp*Z(P,T)|  <  L  for  lul  <  £,  It  -  Tgl  <  T,  p  e  P^T)  .  (2.37) 

m  i 

It  follows  that  we  may  unambiguously  denote  the  two  components  of  the  cylinder 
(2.34)  cut  out  by  by  C+(T),  c“(T),  according  to  whether  that  component 

contains  H+  or  H“,  respectively.  For  x  in  the  cylinder  (2.34)  and 

Tg  -  2T  <  T  <  Tg  +  2T  we  define 

D(x,T)  -  ±  dis(x,ZT)  , 
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the  distance  being  the  distance  within  the  cylinder  (2.34)  and  +  being  used 
in  C+(T),  -  in  C~ (T) .  Then  consider  the  function  D(Xq  -  y(T),T).  Fro*  the 
continuity  of  D(x,T)  (the  proof  of  which  we  omit  here,  but  it  is  not  really 
difficult  to  establish)  it  follows  that  D(Xg  -  y(T),T)  is  a  continuous 
function  of  T.  From  (2.35),  (2.36)  it  follows  that  D(Xg  -  y(T),T)  is 
positive  for  Tfl  -  2t  <  T  <  TQ  -  t,  negative  for  +  r  <  T  <  Tq  +  2t.  Fro* 

the  intermediate  value  theorem  there  exists  at  least  one  T  in 
Tq-t<T<T0  +  t  such  that  Dgtxg  "  fro*  which  it  follows 

that 

xQ  -  y(T)  e  zT 

for  such  T  and,  as  indicated  earlier,  this  shows  that  there  exists  w(t)  a 
solution  of  (2.14)  such  that 

w(0)  -  w(T) 

and  we  have  the  result  for  the  system  (2.14). 

We  indicate  only  briefly  how  the  result  is  extended  to  the  system  (2.1), 

i.e. , 

w  ■  g(w)  +  H(w)u  .  (2.1r) 

The  main  complication  is  that  when  we  let 

y  -  g(y)  +  H(y)u,  y(0)  -  xQ  ,  (2.38) 

then,  in  order  that  w  -  y  +  z  should  be  a  solution  of  (2.1),  we  must  have 

*  -  g(y+s)  -  g(y)  +  (H(y+s)  -  H(y))u  (2.39) 

so  that  small  solutions  s(t)  of  (2.21)  do  not  approximately  satisfy  (2.23)* 
but  rather  the  equation 

£■  [G(y(t) )  +  Gu(t,y(t) )) C  (2.40) 


where 


is  an  n  x  n  matrix  function  of  t,  x  which#  for  each  x#  is  a  linear 
function  of  u(t).  It  is  not  hard  to  show  that 

K»u(t,x)l  <  ylu(t)  I#  t  e  [0,T]  #  x  6  C  ,  (2.41) 

where  C  is  a  compact  subset  of  0  including  a  neighborhood  of  the  periodic 
solution  x(t)  and  y  is  a  positive  constant.  If  we  denote  the  fundamental 
solution  matrix  for  (2.40)  which  reduces  to  the  identity  at  t  *  0  by 
*u(t)#  then 

$u(t)  -  G(y(t))4u(t)  +  Gu(t,y(t))*u(t) 
and  the  variation  of  parameters  formula  applies  to  give,  with  %{ t)  as  in 
(2.24), 


®u(t)  -  *(t)*u(Q)  +/J  ♦(t,8)Gu(s,y(s))*u(s)ds 
-  $(t)  +  ^(t,s)Gu(s,y(s) ) ®u(s)ds 

from  which,  using  (2.40),  a  very  easy  argument  shows  that 

lim  4  (t)  ■  4(t)  . 

BuB  _  ♦  0  U 

I-»E0'Ti1 

Then  all  of  the  arguments  set  forth  earlier  for  (2.14)  apply  with  4(t) 

replaced  by  4  (t),  provided  lul  is  sufficiently  small,  to 

U  I.IO.T.J 

ul  1 

establish  again  the  existence  of  p  and  T  so  that  the  solution  w(t)  of 
(2.1)  with  w(Q)  -  Xq  +  p  satisfies  (2.11).  With  this  we  may  regard  the 

proof  of  Theorem  2.1  to  be  complete. 

The  next  step  is  to  establish,  rather  easily,  a  certain  localisation 
result  for  the  periodic  solutions  w(t)  resulting  from  control  functions 
u(t)  with  luB  ,  small.  Let  us  note,  as  in  (2.19)  earlier,  that, 

given  any  ^  >  0,  we  can  assume 
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ly(t)  -  x(t)  1  <  5,  t  e  [o»T1  ]  , 

where  y(t)  is  now  the  solution  of  (2.38),  by  taking  lul  ,  <  e,  e 

V-V 

sufficiently  small.  Then,  since  x(0)  ■  x(Tq),  we  can  further  assume,  given 
&  >  0,  that 

ly(0)  -  y(T)  I<A,T0-t<T<Tq  +  t  ,  (2.42) 

provided  both  e  and  t  are  taken  sufficiently  small. 

Now  consider  the  map  2(p,T)  »  s(p,T)  -  p  with  z(p,t)  *  z(t)  the 
solution  of  (2.39)  satisfying 

z(0)  ■  p  6  P  • 

Since  3(z(p,T)  -  p)/3p  is  the  restriction  of  the  map  4^(T)  -  1  to  P,  and 
since  p^  8  P  is  (modulo  scalar  multiplications)  the  only  null  vector  of 
4(T)  -  I,  it  follows  that  there  are  positive  numbers  p  and  y  such  that 
for  lul  <  e, 

V°'V 

lz(T)  -  z( 0)  I  -  lz(p,T)  -  pi  >  yip  I 
for  Ip  I  <  2p,  Tq  -  t  <  T  <  Tq  +  t.  Then,  by  taking 

A  <  UP 

in  (2.42)  (which  may  involve  further  restriction  of  e  and  t)  we  see  that 
we  cannot  have 

y(T)  +  z(T)  ■  w(T)  ■  w(0)  »  y(0)  +  z(0)  (2.43) 

for  z(0)  ■  Xq  +  p,  p  0  P,  p  <  Ip  I  <  2p,  since  (2.43)  would  give 

y(T)  -  y(0)  -  z(0)  -  z(T) 

which  is  impossible  since 

ly(T)  -  y ( 0 )  I  <  A  <  yp  <  ylpl  <  lz(T)  -  z(0)l 

for  such  p. 
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Equally  well,  the  proof  of  Theorem  2.1  shows  that  for  small  Ip*,  p  e  P, 


and  appropriately  selected  T  >  0,  no  solution  w(t)  originating  at 


w(0)  ■  xn  +  p  and  corresponding  to  a  control  u  with  lul 


<  e,  e 


ir[0,T.] 
m  1 


sufficiently  small,  can  have  a  period  T  with  T  in  the  ranges  T^  -  2t  <  T 
<  tq  -  T,  tq  +  t  <  t  <  Tq  +  2T.  We  can  put  all  of  this  together  in  the 
following  "localization"  result. 

££2E2-i_ *  —  e  is  sufficiently  small  we  can  find  positive  t  and 


P  such  that  when  lul 


Bd  1 


(i)  equation  (2.1)  has  a  solution  w(t)  periodic  with  period  T  and 
with  w(0)  ■  x(0)  +  p, 

(p,T)  e  N(P,T)  -{pep,  Ipl,  TQ  -  T  <  T  <  TQ  +  T> 

(ii)  equation  (2.1)  has  no  solutions  of  period  T  and  Initial  state 


w(0)  **  Xg  +  p  if 

(p,T)  e  N(2P,2T)  -  N(P,T)  . 

The  localization  result  is  quite  important  in  the  study  of  the  optimal 

control  problem  introduced  below  because  it  enables  us  to  single  out  a  set  of 

triples  x,  u,  T  which  may  be  confined  to  a  bounded  region  simply  by  talcing 

lu*  _  sufficiently  small,  rather  than  by  introducing  further, 

L2[0,T] 

extraneous,  constraints  into  the  problem. 

It  seems  fairly  clear  that  Theorem  2.1  and  Proposition  2.2  can  be 

3f 

extended  to  the  general  system  (1.11)  provided  that  (x,u)  remains 
uniformly  bounded  for  x  e  C  C  0,  u  e  E™,  where  C  is  a  compact  subset  of 
En  containing  the  trajectory  x(t),  0  <  t  <  TQ,  in  its  interior. 
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We  proceed,  now,  to  introduce  and  study  the  optimal  control  problem 
referred  to  in  Section  1.  For  the  moment  we  replace  the  general  cost 
integrand  w(x,u)  of  (1*12)  by  a  less  general  expression,  augmented  by  a 
parameter  o, 

* 

W(  o,x)  +  u  Uu  • 

It  should  not  be  difficult  to  extend  the  work  to  more  general  integrands 
w(  o,x,u) . 

Let  the  system  (2.1),  i.e., 

x  -  g(x)  +  H(x)u,  x  6  En,  u  8  E® 

have  the  properties  set  forth  preparatory  to  and  in  Theorem  2*1  and  let  x(t) 
be  a  periodic  solution  with  least  positive  period  TQ  of 

x  ■  g(x) 

having  the  properties  developed  earlier*  Let  U  be  a  positive  definite 
m  x  m  matrix  and  let  W(  o,x)  be  a  continuous  function  of  o,x  for  o  >  0 
and  x  e  0  <=  Rn  such  that 


W(0,x( t) )  = 

o,  t  e  [o,tq]  , 

(2.44) 

W(  0,x)  >  0, 

a  >  0,  x  e  0  , 

(2.45) 

W(  o,x(t) ) 

4  0,  O  >  o,  t  e  [0,tq]  . 

(2.46) 

For  each  trajectory,  control  and  period  triple  w,  u,  T  with  w(t),  u(t) 
defined  on  the  interval  [0 ,T^ 3 ,  and  w(0)  ■  w(T),  0  <  T  <  T ^ ,  we  define  the 
cost  functional 

J(0,w,u,T)  -  ^  (W(cr,w(t))  +  u(t)  Uu(t)]dt  .  (2.47) 

We  require  T1  >  Tq  and  we  denote  by  M  the  sot  of  trajectory,  control  and 

2 

period  triples  w,  u,  T,  defined  for  0  <  t  <  T, ,  u  6  L  I0,TJ ,  such  that 

I  B  1 

w,  u  satisfy  (2.1)  on  tO,T^] ,  0  <  T  <Tj,  and 

w(0)  ■  w(T)  . 
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There  nay  be  several,  or  even  infinitely  many  values  of  T  corresponding  to  a 
given  pair  w,  u.  Theorem  2.1  shows  that,  given  the  existence  of  the  periodic 
solution  x(t),  with  period  Tn,  corresponding  to  the  control  u(t)  =  0 


A 


with  appropriate  assumptions  on  ®(Tg),  M  includes  at  least  one  triple  w,  u. 


T  for  every  u  e  L  [0,T.]  with  lui  _ 

in  i 


sufficiently  small.  We  make  M 


into  a  metric  space  by  defining  the  distance  function 

V  V  V  V  V 

d(w,u,Tj  w,u,T)  *  d  (R(w),R(w))  +  lu-u* 

H  l  [0/r,] 


+  T  -  T 


where  R(w)  *  (w  =  En  |  w  *  w(t)  for  some  t  e  [0,T  ])  and  djj  is  the  usual 
Haussdorff  metric  on  compact  sets. 

Property  (2.46)  of  the  function  W  shows  that  if  o  >  o 

J(o,x,0,T0)  2  JQ(o)  >  0  , 

and  the  continuity  of  W  with  respect  to  0  shows  that 

lira  J  (o)  »  0  . 

o+O  0 

For  o  >  0  we  define  M„  to  be  the  subset  of  M  for  which 

o 

J(0,w,u,T)  <  JQ(0)  . 

If  w,  u  T  e  Mo,  it  is  clear  that 


which  implies  that 


£  u(t)*Uu(t)dt  <  JQ(0)  , 


2  T  J0(O)  Vo(0) 

/J  lu(t)»  dt  <  — - -  <  — - 

1  1 


(2.48) 


(2.49) 


when  yi  is  the  smallest  eigenvalue  of  the  symmetric  positive  definite 
matrix  U. 


The  localization  result  expressed  in  Proposition  2.2  shows  that  the  set 


I 


■i 


t 


of  all  w,  u,  T  in  M  for  which  lul  <  e  nay,  if  e  is 

V°'V 

sufficiently  snail,  be  decomposed  in  a  natural  way  into  two  components <  those 

lying  in  N(p, t)  and  those  lying  outside  N(2p,2t),  there  being  a  triple  w, 

2 

u,  T  in  N(  p,  t)  for  every  u  in  L  [0,T.]  with  lul  _  <  e.  The 

m  i 

importance  of  this  result  lies  in  the  fact  that  it  provides  us  with  a  priori 

bounds  on  a  certain  subset  of  the  periodic  orbits  associated  with 
2 

u  e  1  of  small  norm  without  the  necessity  of  imposing  any  additional 

constraints. 

Theorem  ^.3.  Let  e  >  0  be  sufficiently  small  and  let  p,  t  be  such 
that  Proposition  2.2  applied  for  lul  <  e.  Let  c  >  0  be  small 

L*[°,Til 

enough  so  that  (cf.  (2.49)) 


J0(o>  <  T  •  <2.50) 

.  A  A  A 

Then  there  exists  at  least  one  element  w,  u,  T  in  N(p,  x)  such  that 
J(c,w,u,T)  <  J(c,w,u,T),  w,u,  T  e  N  (  p,  t)  . 

Proof .  The  Caratheodory  regularity  theory  referred  to  earlier  allows  us  to 
see  that  if  e  >  0  is  sufficiently  small  and  w,  u,  T  is  in  N(p, x)  with 

lul  _  <  e,  then  there  exists  B(  e)  >0  such  that 

L  [  °  ,  T .  ] 
in  i 


tw(t)  -  x(t)  I  <  B(  e),  t  e  [0,^]  . 

Moreover , 

lim  B(  e)  ■  0 
e-K) 

In  particular,  then,  there  is  some  M  >  0  and  a  closed  subset 
C  C  O  D  {w  |  Iwl  <  M)  such  that 

w(t)  e  c,  t  e  [o,t^]  , 


4 
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for  such  w,  u,  T.  Using  the  Schwarz  inequality  we  quickly  see  that 


where 


lw(t)  -  w(t+$t)*  -  */£+6t  (g(w(s))  +  H(w(s))u(s))ds< 

<  g^t  +  h1  /^+St  *u(s)*ds 

<  g  6t  +  h,  (/"+6t  *u(s)l2ds)1/2  (/"+6t  1  ds)‘ 

lit  » 


<  g  6t  +  h  £ 


g  *  sup  lg  (w)  *  , 

1  wee 


h  *  sup  *H(w) *  . 

wee 


Since  (2.50)  is  true,  (2.49)  shows  that  we  may  assume  a  minimizing  sequence 
wk'  uk'  Tk  such  that  (starting  with  k  =  2  to  avoid  confusion  with 


already  defined) 


wk,  uk,  Tk  e  N(P,T),  k  -  2,3, • , 


T.J„(0> 


S'  2 

Lm[0'T1] 

m  * 


<  C,  k  -  2,3, • • • i 


lim  J(o,w  ,u  ,T  )  -  inf 
k-*»  w,u,TeN(P,T) 


inf  J(o,w,u,T)  .  (2.51) 


From  the  boundedness  and  equicontinuity  of  the  wv,  the  boundedness  of  the 

2  _  .  Vo(0\ 

T,.  and  the  weak  compactness  of  (u  6  I.  [0,T  )  |  lu<  .  *  JJ  '  w® 

k  "  1  lT[0,T.] 

ft  1 

conclude  that  we  can  find  w,  u,  T  in  N(P,x)  and  a  subsequence  of 
(wk,uk,Tk)  which,  for  convenience,  we  will  still  call  ^wk'uk,Tk^  such  that 


9 


(2.52) 


lim  (sup  lw.(t)  -  w(t)l)  -  0 
k+-  teT1 


lia  T  «  T  (2.53) 

k+® 

and  converges  weakly  to  u  in  L2 (0,T) .  Combining  (2.52)  and  (2.53) 

it  is  clear  that 


T  A 

~  /„k  «(o.wk(t))dt  -  ~  f*  W(o,w(t))dt  . 
k‘H,B  k  T 


From  the  weak  convergence  of  (vl  }  to  u  in  L2[0,T1  we  immediately  have 

k  ml 

the  weak  convergence  of  {u^  to  u  in  L2[0,T].  Then,  since  for  any 

V  e 


T  T 

l/$k  V(t)\(t)dt|  <  (J$k  luk(t)l2dt)1/2 

x  (/Ik  'v(t)l2dt)1/2  <  elv» 

V*'V 


(or  L2[T.  ,T]  if 
m  k  . 


T^  <  T)  and,  as  is  well  known 


lim 

T, 

k 


f  ( t )  dt  =-  0 


for  any  integrable  f(t),  we  conclude  that  if  the  uk(t)  are  redefined  to  be 
zero  in  [T^,T11,  k  *  2,3, •••  and  u(t)  is  redefined  to  be  zero  in  [T,T1J, 
it  will  stiil  be  true  that  (u^)  converges  weakly  to  Then,  using  a 

result  in  [H]  (Theorem  8,  p.  209)  we  know  that 


/s  X 

/J  u(t)  Uu(t )dt  <  lia  J*  uR(t)*Uuk(t)dt 
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and  we  see  that 


J(  0, w,u,T) 


<  lim  J(  o,wk,a  ,T  ) 
k-H*> 


and  hence,  from  (2.51),  that 

A  A  A 

J(o,w,u,T)  -  inf  J(  o,w,u,T) 

w,u,TeN(  p,  t) 


«  lim  J( 
k+« 


and  the  theorem  is  proved. 

It  will  be  seen  in  the  course  of  the  discussion  in  Section  3  that  the 
above  optimal  control  formulation  is  not  adequate  for  all  purposes.  In  some 
cases  we  need  to  take  a  fixed  cost  functional 

J(x,u,T)  -  ~  [W(x( t) )  +  u(t)  Uu( t) ] dt 

where,  assuming  the  system  (2.1)  now  augmented,  as  in  Section  1,  with  a 
parameter  0,  viz.: 

x-f(x,6,u)  (2.54) 

or 

x  *  g(x,0)  +  H(x,0)u  ,  (2.55) 

we  suppose  that  W(x(t))  s  0  on  a  T-periodic  solution  x(t)  of  the 
"uncontrolled"  system  (2.54)  or  (2.55)  with  u  =  0  and  0*9,  where  6  is 
some  minimal  value  of  the  parameter  p,  and  that 

W(x)  >  0,  x  i  {x( t)  |  0  <  t  <  I)  . 

The  optimal  solution  for  0*9  is  clearly  x(t),  0  <  t  <  T.  One  wishes  to 
ttttdify  Theorem  2.3  to  establish  the  existence,  locally,  of  an  optimal  solution 
for  6  in  some  neighborhood  of  6. 
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The  only  modifications  that  are  at  all  significant  here  involve 
modification  of  the  localization  result.  Proposition  2.2,  so  that  it  applies 


for  lul  sufficiently  small  and  d  sufficiently  near  6  and 

i/tO.T,] 
m  i 

replacement  of  the  a_  priori  bound  ( 2 . 48 )  on  u  by 

T 

£  jj  u(t)*Uu(t)dt  <£-  /QW  W(x  (t))dt 

U 

where  x  (t)  is  some  periodic  solution  of 
Vi 

x  (t)  *  f(x  <t),  vi, 0 ) 

VJ  vi 

lying  near  the  periodic  solution  x(t)  with  period  T  ^  near  T  -  the 

existence  of  which  is  established  much  as  in  Theorem  2.1,  except  that  the 

proof  can  be  greatly  simplified  now,  using  the  implicit  function  theorem 

since,  regarded  as  a  control,  p  is  now  (trivially)  continuous  in  t.  In 

fact,  x  and  T  are  unique  in  this  case.  The  rest  of  the  argument  is 
UP 

almost  word  for  word  the  same  as  in  Theorem  2.3  above  and  will  not  be  repeated 
here. 
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3.  Necessary  Conditions  for 


Euimiim 


CgjPuUtioBgl  Considerations 


Zn  the  present  section  we  consider  the  system 

x  -  f(x,u),  x  e  0  c.  e",  u  e  E®  ,  (3.1) 

wherein  it  is  assumed  that  f(x,u)  is  continuously  differentiable  with 
respect  to  both  x  and  u  over  the  range  indicated.  He  suppose  that  x(t) 

A 

is  a  periodic  solution  of  (3.1 ),  with  least  positive  period  T,  which 

A  A 

corresponds  to  the  control  u(t)  on  [0,T].  For  the  moment  we  make  no 
further  assumptions  on  f  but  restrict  attention  to  piecewise  continuous  u 
and  piecewise  continuous  variations  6u  from  u,  but  fairly  minor 
modifications  allow  us,  in  the  case  of 

x  -  g(x)  +  H(x)u,  x  e  0  C  En,  u  B  E®  ,  (3.2) 

the  system  treated  in  Section  2,  to  also  admit  the  possibility  that  u  and 
the  variations  6u  are  just  square  integrable. 

A  number  of  authors  have  considered  necessary  and  sufficient  conditions 
for  periodic  systems  ([I],  [J] ,  [K]  ,  [L] ) .  He  do  not  develop  the  conditions 
here  because  they  are  new  -  they  are  not  -  but  to  allow  us  to  make  appropriate 
reference  later  in  this  section.  He  seek  necessary  conditions  in  order  that 
the  control  u  should  afford  a  local  minimum  (at  least)  for  the  cost 
functional 

J(x,uT)  *  v( x( 0 ) )  +  £  w(x(t) ,u(t) )dt  ,  (3.3) 

where  v(x)  and  w(x,u)  are  continuously  differentiable  in  x,  (x,u) 
respectively,  for  x  e  OC  e”,  u  e  E1".  In  (3.3)  it  is  assumed  that  x(t)  is 

A  /\ 

a  periodic  solution  of  (3*1)  having  period  near  the  period  T  for  x(t): 

It  -  4|  <  t,  t  >  o  , 

that  x(t)  lies  near  x(t)  in  the  sense  that 

lx(t)  -  x(t)»  <  €,  0  <  t  <  sup(T,T) 
for  some  c  >  0  and  that 
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Iu(t)  -  u( t)  I  <  4,  0  <  t  <  sup(T,T)  , 

for  im  6  >  0.  In  the  case  of  (3.2)  this  can  ba  ralaxsd  to 

lu  -  ul  ,  <  6,  T  >  nax{T,T)  • 

L210/T  ]  A 

Lot  <5u(t)  bs  piecewise  continuous  for  0  <  T^,  T^  >  T.  ffa  considsr 

controls 

u(t)  -  u(t)  +  o  flu(t),  a  rsal  ,  (3*4) 

A  A 

where  u(t)  is  axtandad  outsida  [0,T]  by  pariodicity  as  raquirad* 

We  supposa  the time  parameter  noraalicad  in  such  a  way  that  u(t)  is 

A 

continuous  at  t  -  0  ( aquivalantly  at  T)  and  wa  sat 

F(t)  -  F(x(t),  u(  t ) )  -  ~  ( x( t) ,  u( t) ) ,  0  <  t  <  T  .  (3.5) 

As  aarliar  in  tha  paper,  wa  danote  by  6(t,s)  tha  matrix  solution  of 

-  F(t)*(t,s),  «(s,s)  -  I  , 
and  wa  abbraviata  *(t,0)  by  4(t) . 

Tha  variational  thaory  for  a  controlled  periodic  solution  x(t)  is 
somewhat  different  from  that  for  an  uncontrolled  periodic  solution  x(t)  as 
discussed  aarliar  in  this  paper*  If  wa  let 

p  -  f(x(0),  u(0))  (3.6) 

A 

it  is  no  longer  necessarily  true  that  p  is  an  eigenvector  of  *(T) 
corresponding  to  an  eigenvalue  1^*1.  To  saa  why  this  is  so,  consider, 
a*g«,  tha  case  where  u(t)  is  periodic  and  of  class  C1  on  (0,T]«  Let 

p(t)  -  x(t)  •  f(x(t),  u(t)J,  0  <  t  <  T  . 

Differentiating  (3.1)  gives 

p  -  F(t)p  +  H(t)u(t)  ,  (3.7) 

where 

H(t)  ■  H(x(t) ,  u(t))  «  (x(t),  u(t))  ,  (3.8) 


so  that 


p(t)  -  *(t)p(0)  +/J  t(t,s)H(s)u( s)  ds  . 

A  A  A  A 

Since  p(T)  -  x(T)  -  x(0)  -  p(0 )  «  p,  we  have 

A  • 

(I  -  *(T))p  -  /J  4’(T,s)H(s)u( s) da  (3.9) 

A 

and  p  is  an  eigenvector  of  $(T)  corresponding  to  the  eigenvalue  1  if  and 
only  if  the  right  hand  side  of  (3.9)  turns  out  to  be  zero*  Of  course,  in  the 
uncontrolled  case  the  right  hand  side  is  automatically  equal  to  zero. 

He  are  going  to  proceed  under  the  following 
Assumption »  Either  (i)i  p,  given  by  (3.6),  is  the  unique  eigenvector  of 

A  A 

♦(T)  corresponding  to  the  eigenvalue  1,  which  is  simple,  or  (ii)»  *(T) 

has  no  eigenvalue  equal  to  1 . 

A 

It  is  not  particularly  difficult  to  treat  the  case  where  9(T)  may  have 
a  simple  eigenvalue  1  with  corresponding  eigenvector  different  from  p  but 
we  shall  not  do  so  here. 

In  case  (i)  the  study  of  perturbed  periodic  solutions  corresponding  to  a 
small  perturbation  5u( t)  from  the  control  u(t)  is  just  Theorem  2.1.  In 
case  (ii)  one  can  establish  the  following 

A 

Proposition  3.1 i  If  $(T)  has  no  eigenvalue  equal  to  1  then, 
corresponding  to  each  piecewise  continuous  perturbed  control 

u(t)  “  u(t)  +  o  fiu(t) 

with  ( ot|  sufficiently  small,  there  is  a  unique  perturbation  5x(0)  of  the 
initial  state  so  that  the  solution  x(t)  of.  (3.1)  corresponding  to  the 
control  u(t)  and  the  initial  state  x(0)  -  x(0)  +  &c(0)  remains  periodic 

A 

with  period  T. 

We  omit  the  proof.  A  comparable  result  can  be  obtained  for  (3.2)  with 
6u  in  L2I0,T]. 
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A 


To  proceed,  we  consider  a  perturbed  control  (3.4),  which  we  take  to  be 


piecewiae  continuous  in  this  discussion,  and  we  let 

x(t,a)  ■  x(t)  +  a  fix(t)  +  o(|a|),  |«|  +  0 


(3.10) 


be  a  perturbed  periodic  solution  near  x(t)  (uniqueness  fails  under  case  (ii) 
of  our  Assumption  even  with  piecewise  continuous  controls)  with  period 

T(a)  -  t  +  a  6t  +  o(|o|),  |o|  ♦  0  . 

In  (3.10)  fix(t)  is  a  solution  of  the  variational  equation  (cf.  (3.5),  (3.7), 


(3.8)) 


fix(t)  -  F(t)fix(t)  +  H(t)fiu(t) 


(3.11) 


for  0  <  t  <  T,  and  the  equation  is  extended  outside  this  range,  as  required, 

using  the  T  periodicity  of  x(t)  and  u(t). 

Denoting  the  gradients  of  v,  w  with  respect  to  x,  (x,u), 

3  v  3  w  3w 

respectively,  by  g^,  -g^,  g^#  and  treating  these  as  row  vectors,  it  may  be 
seen  that 

J(x(  *,o)  |  u(  *,<*),  T( a) )  -  J(x,u,T) 


+  (|j  (x(0) )fix(0)  +  i  /f  l|j  <x(t),  u(t))«x(t) 

T 

a  a 

+  g^  (x(t),  u(t) )$u(t)] dt  +  C”  w(**t*»  u(t) )dt 


+  —  w(x(0) ,  u(0) ) )6t)  a  +  o(|a|),  a  ♦  0 


(3.12) 


wherein  we  have  used  periodicity  to  see  that 

w(x(T) ,  u(T) )  -  w(x(0),  u(0))  . 

Now  the  variational  derivatives  fiu,  fix  and  fiT  are  not  independent; 
the  last  two  are  determined  by  fiu  and  the  normalization  of  fix(0).  In  order 
to  be  able  to  express  the  cost  variational  derivative  explicitly  in  terms  of 
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«u,  rather  than  Implicitly  through  $x  and  $T,  one  introduce*  a  solution 
X(t)  of  the  "adjoint  system” 


X(t)  -  -F(t)*X(t)  +  ^  |j  (x(t),  u(t))  .  I 

T 

The  boundary  conditions  will  be  specified  shortly*  We  find  then  that 


X(T)*«x(T)  -  X(0)*6x(0)  -  Jj  ~(X(t)*6x(t))dt 


A  a 

(Cf.  (3.11),  (3.13))  -  /?  ((-X(t)*F(t)  +  -I*  (x(t),  u(t))«x(t) 


+  X(t)  (P(t)6x(t)  +  H(t)fiu(t)))dt 


(3.13) 


A  A  , 

/J  X(t)  H(t)fiu(t)dt  *  X'  ^0  dx  ******  u(t))6x(t)dt  . 

T 


To  first  order,  the  periodicity  requirement 


(3.14) 


x(T  +  $T)  -  $(0) 


may  be  replaced  by  the  condition 


and  then,  since 


A  fl  A 

$x(T)  +  —  (T)6t  *  $x(0) 

at 


(T)  -  f (x(T) ,  u(T) ) 


we  have 


f(x(0),  u(0)) 


5x(T)  +  f(x(0),  u(0))6t  ■  6x(0)  . 


(3.15) 


Using  (3.14)  and  (3.15)  in  (3.12)  we  see  that 


M  W- 


33  <xC,«>,  uC,a),  - 

(x(0)>  -  X(0)*)«x(0)  -  (X(T)*f(x(0),  u(0) ) 

A 

-  —  w(x(0) ,  u(0 ) )  +  “2  W(x(t),  u(t))dt)$T 

T  T^ 

*  .  «  t 

+  1!  [Jr  (x(t),  u(t))  -  X(t)  H(t)]«u(t)dt  .  (3.16) 

T  U 

The  dependence  on  Sx( 0)  is  annulled  by  imposing  the  boundary  condition 

X(T)  -  X(0)  +  (x(0) )  -  0  .  (3.17) 

At  this  point  we  have  to  consider  two  possibilities,  corresponding  to  the 

A 

two  cases  delineated  in  the  above  Assumption.  If  case  (i)  obtains,  *(T)  has 
a  simple  eigenvalue  of  1.  Then  the  fundamental  solution  matrix 

’T(t)  -  (*(t)V1  , 

obeying 

^3—--  -F(t)*'?(t) 

A 

is  such  that  “(T)  has  an  eigenvalue  equal  to  1  and 

X(t)  -  -F(t)*X(t) 

has  a  one  parameter  family  of  T-periodic  solutions  which  we  may  represent  in 

v  A  . 

the  form  0X(t).  Given  a  solution  X(t)  of  (3. 13), . (3. 17) 

X(t)  -  X(t)  +  BX(t) 

also  satisfies  (3.13),  (3.17)  for  every  value  of  6.  Then  we  may  expect  to  be 
able  to  select  0  (with  suitable  non- degeneracy  requirements)  so  that  X(t) 
also  satisfies 
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X(T)  f(x(0),  u(0))  -  £  w(x(0),  u(0)  +  “ T  in  w(x(t)#  u(t))dt  *  0  .  (3.18) 

T 


5,2  '0 
T 


With  X(t)  so  chosen,  we  see  from  (3.16)  that  d7/d<*  -  0  for  all  6u(t) 
just  in  case 

X(t)*H(t)  -  (x(t),  u( t ) )  -  0,  t  8  [0,T]  .  (3.19) 

T 

If  case  (ii)  obtains  we  have  noted  that  we  can  find  perturbed  periodic 
solutions  for  which  St  ■  0.  This  also  annuls  the  second  term  in  (3.16)  and 
enables  us  to  conclude  that  (3.19)  must  hold.  But,  in  case  (ii)  still,  it  is 
also  possible,  for  each  6u  and  a  sufficiently  small  in  (3.4),  to  find 
perturbed  periodic  solutions  x(t,a)  near  x(t)  with  periodic  T  +  <*  $T, 

6t  0  given  in  advance  (this  is  true  because  (cf.  (3.15))  we  can  always 
solve 

(*(T)  -  I)6x(0)  +  f(x(0),  u(0))«T  -  0 

in  case  (ii)).  Assuming  (3.19)  already  established  as  noted,  dJ/da  reduces 
to  the  second  term  in  (3.16)  and  we  conclude  that  condition  (3.18)  must,  in 
fact,  hold  in  this  case  also. 

It  will  be  observed  that  if  (x(0))  ■  0,  (3.17)  becomes 

>x 

X(T)  -  X(0)  -  0 

,  A 

so  that  A(t)  is  periodic  with  period  T. 

It  is  customary  to  refer  to 


H(X(t),  x(t),  u(t),  T) 

-  X(t)*f(x(t),  u( t ) )  -  £  w(x(t),  u( t ) ) 


(3.20) 


as  the  Hamiltonian  for  this  optimization  problem.  The  condition  (3.19)  is 


»H  A..  .  A  A.  .  -  A. 


3u 


(X(t),  x(t),  U(t),  T)  -  0,  t  e  [0,T]  . 


(3.21) 


We  have  the  following  result. 
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Proposition 1 3 » 1 :  H(X(t),  x(t),  u(t),  T)  is  constant  for  t  0  [0,T] 

Proof.  Proa  (3.5),  (3.8),  (3.20)  we  have 


H( X(t) ,  x(t),  u(t),  T)  -  X(t)*£(x(t),  u(t)) 


+  (X(t)  F(t)  (x(t),  u(t)))x(t) 

T 


+  ( X(t)  H(t)  -  ^  ^  (x(t),  u(t)))u(t) 
T 


(using  (3.19),  (3.1),  (3.13)) 


(-X(t)*F(t)  +  *  ~  (x(t),  u(t))]x(t) 
T 


+  IX(t)  F(t)  -  ^  (x(t),  u(t))]x(t)  »  0  . 

T 

When  X(T)  -  X(0),  (3.19)  will  ordinarily  give  ${T)  ■  u(0)  and, 

x(T)  *•  5(0),  (3.18)  becomes 

A 

H(X(t),  5(t),  u(t),  T)  w(x(s),  u(s))ds,  te  [0,T]  .  (3, 


If  we  let 


U(t) 


$  X(t), 


t  e  [0 ,T]  , 


we  have,  for  t  e  tO,T] , 


y(t)  -  -F(t)*u(t)  +  (x(t),  u(t))  , 


(3, 


U(t)*H(t)  -  -|*  (x(t),  u(t) )  -  0 


(3. 


where 


H(u(t),  x(t),  5(t))  -  0  , 


(3. 


H(y(t),  x(t) ,  u(t) )  *  p(t)  f(x(t),  u(t))  -  w(x(t),  u(t))  .  (3. 


since 

21) 

22) 

23) 

24) 

25) 
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(3.26) 


In  this  cage 

J(x,u,T)  =  H(W,(t),x(t),u(t))  +  v(x{ 0 ) ) ,  t  e  [0,T]  . 

Very  little  needs  to  be  changed  in  the  development  which  leads  up  to 
Theorem  3.2  in  order  to  permit  u  to  be  a  function  L2[0,Tj  in  the  case  of 
the  special  class  of  systems  (3.2).  In  fact,  the  class  is  selected  so  that 
with  controls  u(t,a)  as  in  (3.4),  but  u(t),  $u(t)  now  in  L2[0,T],  the 
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resulting  solution  x(t,a)  still  has  the  form  (3*10),  with  5x(t)  satisfying 

( 3. 1 1 ) •  For  general  functions  f(x,u)  it  is  not  easy  to  see  that  this  is  the 

case  without  making  some  rather  technical  assumptions  on  f  to  account  for 

the  possibility  that,  while  I$u<  in  L^[0,T]  might  be  small,  6u(t)  might 

still  assume  arbitrarily  large  values.  The  form  (3.2)  can  be  replaced  by 

df 

various  boundedness  assumptions  on  but  we  elect  not  to  pursue  this  here. 

A  o  A 

The  time  variable  in  the  case  of  (3.2)  with  u  in  L  [0 ,T]  should  be 
normalized  in  such  a  way  that  0  is  a  Lebesgue  point  for  u,  with  the  effect 

.A 

that  rrr  (0)  is,  indeed,  equal  to  g(x(0)  +  H(x(O))0(O),  so  that  (3.15) 
at 

appropriately  modified,  still  makes  sense.  (Of  course  u(t)  turns  out  in 
nearly  all  cases  to  be  smooth  and  periodic.) 

It  is  not  generally  very  easy  to  prove  the  existence  of  solutions  of  the 
equations  constituting  this  set  of  necessary  conditions.  However,  under  the 
circumstances  Which  correspond  to  Theorem  2.3  it  is  possible  to  p^ove  the 
existence  of  a  unique  solution  (in  a  certain  sense)  for  small  values  of  the 
parameter  o,  provided  we  adjoin  to  (2.44),  (2.45),  (2.46)  the  further 
condition  that 

(0,x(t) )  =  0,  t  e  [0,Tq]  (3.30) 

and  the  function  v(x)  appearing  in  (3.3)  satisfies  certain  requirements, 
about  which  we  will  have  more  to  say  below.  Let  us  notice  first  of  all  that 
when  v(x)  =  0  there  is  no  possibility  of  a  unique  minimum  for  the  cost 
functional  (3.3)  under  the  stipulated  circumstances  because  translation  by  a 
fixed  5t  along  the  periodic  optimal  trajectory  x(t),  W(t)  ((3.27)  shows 
U(t)  to  be  periodic  when  v(x)  =  0)  yields  a  new  trajectory  with  the  same 
cost  as  well  as  a  new  set,  strictly  speaking,  of  solutions  of  the  necessary 
conditions.  This  sort  of  non-uniqueness  is  not  very  interesting,  of  course. 
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and  it  is  customary  and  convenient  to  rule  it  out  by  imposing  an  appropriate 
normalizing  condition  on  the  initial  state  x(0).  Letting  (cf.  Section  1) 

p  -  x(  0 )  -  f(x(0),0) 

one  can  require/  for  example ,  that  a  nearby  optimal  x(t)  should  satisfy 


p*($(0)  -  x(0))  -  0 


(3.31) 


thereby  confining  the  initial  state  x(0)  to  an  ( n-1 ) -dimensional  hyperplane 
transverse  to  the  nominal  orbit  x(t).  Computationally  it  is  preferable  not 


to  enforce  (3.31)  as  an  explicit  constraint  but/  instead/  to  use  for  v(x)  a 


function  such  as 


v{ x)  -  <p*(x(0)  -  X(0)))2 

-  (x(0)  -  x(0))*pp*(x(0)  -  x(0 ) )  . 


(3.32) 


In  obtaining  the  least  possible  cost  for  (3.3)  one  automatically  obtains  the 
periodic  x(t)  which  satisfies  (3.31)  and,  in  the  end,  —  (x(0))  “0  so 
that  (3.27)  applies  to  show  jj(t)  to  be  periodic. 


Provided  a  correctly  constructed  normalizing  v(x)  is  used  as  indicated 
above,  it  is  possible  to  use  the  implicit  function  theorem  to  construct 
x(t),  u(t),  near  x(t),  0,  satisfying  the  necessary  conditions  set  forth  in 
Theorem  3.2.  One  takes  x{0),  y(0)  to  lie  in  a  small  neighborhood  of  x(0), 
0.  Then,  via  solution  of  the  differential  equations  (3.1),  (3.13)  with  u(t) 


determined  by  (3.19),  the  vectors 


A  A  A 

x(T)  -  x( 0 )  , 


(3.33) 


A  A  JVif  A 

U(T)  -  M(0)  +  T  -g  (x(0 ) ) 


(3.34) 


and  the  scalar  quantity  (cf.  (3.28)) 


H( u(0) ,x(0) ,u(0) )  +  v(x(0 ) )  -  7  jT  w(x(t) ,u(t))dt  (3.35) 

T 
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I 


are  seen  to  constitute  a  (2n+1)  dimensional  vector  function  of  x(0),  U(0), 

T  and  any  relevant  parameters,  such  as  a  of  Theorem  2.3  or  the  parameter 
V,  as  in  the  discussion  of  the  alternate  problem  following  Theorem  2.3.  The 
Jacobian  of  (3.33),  (3.34),  (3  35)  with  respect  to  x(0),  U(0)  and  T  can  be 
computed  with  the  aid  of  the  variational  equations  and  under  appropriate 
circumstances  (e.g.  the  cost  functional  has  the  form  discussed  in  Theorem  2.3, 
(3.30)  holds  and  v(x)  is  defined  as  in  (3.32))  one  can  show  that  this 
Jacobian  is  nonsingular  when  0  ■  0  (or,  depending  on  the  problem,  when 
U  =■  0 )  and  unique  x ( 0 ) ,  U ( 0 ) ,  T  can  then  be  inferred  for  o  (or  W ) 
sufficiently  small.  We  will  not  go  into  detail  on  this  as  it  will  appear 
elsewhere  and  would  lead  us  much  too  far  astray  here. 

The  computational  version  of  the  above  is  the  simple  Newton's  method  for 
solving  the  equations  obtained  by  setting  (3.33),  (3.34),  (3.35)  equal  to 
zero.  Implementation  requires  numerical  integration  of  the  variational 
equations  over  an  interval  (0,Tk3 ,  where  Tk  is  the  latest  approximation  to 
the  period  $  in  order  to  be  able  to  compute  the  Jacobian  of  (3.33),  (3.34), 
(3.35)  with  respect  to  the  current  initial  conditions  for  xfc,  and  the 
period.  Just  as  noted  above,  it  can  be  established  that  this  procedure  is 
locally  convergent  in  certain  cases.  However,  this  method,  while 
theoretically  useful  and  appealing  in  its  simplicity,  suffers  from  a  number  of 
disadvantages  as  a  computational  procedure.  The  necessity  of  integrating  the 
variational  equations  to  obtain  the  Jacobian  matrix  is  one  such  disadvantage 
to  begin  with,  but  more  serious  is  the  frequent  numerical  instability  of  the 
procedure.  In  most  cases  the  periodic  solution  x(t),  corresponding  to 
u(t)  «  0,  that  one  begins  with,  is  asymptotically  stable.  The  corresponding 
variational  system  is  a  time  varying  periodic  system  whose  characteristic 
exponents  have  negative  real  parts.  But  the  dual  variational  system  for  the 
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« 


adjoint  variable  M  (or  A)  has  characteristic  exponents  which  are  the 
negatives  of  those  of  the  variational  system  for  x(t).  Thus  the  variational 
system  for  M  (or  A)  has  solutions  which  may  grow  very  rapidly.  This  means 
that  it  is  often  very  difficult  to  accurately  compute  the  part  of  the  Jacobian 
corresponding  to  the  equation  (3.34).  This  has  been  a  serious  limiting  factor 
in  our  computational  experience  thus  far. 

While  theoretically  less  rapidly  convergent,  the  steepest  descent  method 
offers  much  in  the  way  of  improved  performance  from  the  standpoint  of 
numerical  stability,  primarily  because  the  state  equations  and  adjoint 
equations  are  integrated  in  opposite  directions  to  take  advantage  of  their 
differing  stability  characteristics.  We  will  describe  briefly,  one  step  of 
such  a  procedure. 

We  shall  suppose  that  x^t) ,  u^(t)  form  a  periodic  trajectory  -  control 
pair  for  (3.1)  with  period  T^.  As  we  do  not  require,  ab  initio,  any 
continuity  for  u^,  the  periodicity  refers  to  the  trajectory  x^tt).  With 
x(t)  »  xk(t)  available,  one  proceeds  to  compute  a  solution  W^ft)  of  (3.13). 
Corresponding  to  the  two  cases  admitted  under  the  Assumption  above,  one 
follows  two  routes.  If  case  (i)  applies,  |i  ( t )  is  determined  so  that  it  also 
satisfies  (3.28).  Then  setting 

fiu^ft)  -  H(t)  Mk(t)  -  |jj*  ( x( t )  ,u ( t ) )  (3.36) 

gives  (cf.  (3.16))  <17/ da  <  0  (if  (3.23)  does  not  already  hold  for  x^,  W^). 

Por  small  positive  a  we  let 

uk+1(t)  -  u^t)  +  a  Su^t),  t  e  [0,Tfc]  . 

Extension  of  Su^tt)  outside  [0,^],  where  required,  can  be  accomplished 
from  (3.36)  and  we  may  assume  this  has  already  been  done  for  uk(t).  Then  we 
determine  5x^(0)  and  St^  so  that  xfc+1(t),  the  solution  of  (3.1)  with 


initial  stata  *k(0)  +  6x^(0)  and  u(t)  =  uk+1(t)  satisfies  x^^lT^  + 

”  x]c+i(T]t+i)  "  xk+1^*  **  a  >  0  is  sufficiently  small  one  can  show 

that  the  cost  associated  with  x)<+1 ,  uk+1  will  be  smaller  then  that 
associated  with  x^,  u^.  If  case  (ii)  applies  is  onty  required  to 

satisfy  (3.27),  6u^  is  selected  as  in  (3.36),  and  we  let  6t^  “  a  times  the 
quantity  on  the  left  hand  side  of  (3.28)?  we  will  again  see  from  (3.16)  that 
d3/da  <o.  This  defines  the  new  period  t^+1  -  +  6t^  and  we  proceed,  as 

we  may  in  case  (ii),  to  find  ,  corresponding  to  the  control  uJc+1 , 

having  precisely  the  period  T^+1 .  Again  the  cost  associated  with  xk+1,  u^+1 
will  be  reduced,  as  compared  with  x^,  u^,  if  a  >  0  is  sufficiently  small. 

The  above  procedure  is  continued  until  the  necessary  conditions  are 
satisfied.  The  integrations  necessary  to  determine  ufc(t)  are  always  carried 
out  in  reverse  time  (as  compared  with  those  for  x^ ( t ) )  for  the  sake  of 
stability.  The  above  description  is  very  rough  and  a  number  of  refinements 
remain  to  be  delineated?  for  example,  how  do  we  treat  the  situation  which 
obtains  when  case  (i)  is  very  nearly,  but  not  quite,  true.  This  is  not  the 
place  to  treat  such  questions. 

While  case  (ii)  may  be  considered  generic  for  our  problem,  case  (i)  does 
arise  in  significant  instances?  e.g.  with  a  cost  functional  (2.47),  satisfying 
(3.30),  the  initial  step  away  from  the  uncontrolled  solution  must  be  made 
under  case  (i)  circumstances. 

An  Example.  Initial  computational  experiments  have  been  carried  out  with  the 
second  order  scalar  system 


with  the  parameter  6  in  the  range  0  <  8  <  .6  with  the  objective  of 
amplitude  suppression.  The  uncontrolled  system  has  a  self-excited  oscillation 
with  amplitude  A(  8)  shown  by  the  solid  curve  A(  8)  in  Figure  3*2,  the  whole 
orbit  for  8  ■  .6  being  shown  by  the  closed  solid  curve  in  Figure  3.1. 

Initially  we  worked  with  a  cost  functional  of  the  form 

£  jj  to(x(t)2  +  (x(t))2)  +  u(t)2]dt  ,  (3.38) 

first  computing  the  uncontrolled  orbit*  which  is  optimal  for  c  *  0  and  then 
increasing  o  in  small  increments.  The  computational  method  used  was  the 
Newton  method  described  earlier.  The  cost  functional  (3.37)  has  the  form 
discussed  in  Theorem  2.3.  It  proved  unsatisfactory,  however.  In  a  very 
narrow  range  of  positive  values  a  controlled  periodic  solution  of  smaller 
amplitude  was  obtained  but  it  "popped",  like  a  bubble,  for  a  "  .023.  The 
word  "popped"  is  used  advisedly  because  we  believe  the  disappearance  of  the 
solution  near  this  value  of  a  is  indeed  due  to  mathematical  causes  similar 
to  those  which  are  operative  in  the  bursting  of  a  bubble  or  the  stretching  of 
a  soap  film;  the  stability  of  the  initial  uncontrolled  solution  manifests 
itself  initially  -  a  finite  amount  of  control  energy  is  required  just  to  move 
away  from  this  initial  solution.  Beyond  a  certain  point,  however,  a  minimal 
cost  solution  of  positive  amplitude  no  longer  exists  -  the  origin  is  the  only 
(degenerate)  solution.  We  are  fairly  well  satisfied  that  the  optimal  solution 
does,  in  fact,  disappear  in  "mid-air"  rather  than  collapse  continuously  toward 
the  origin  -  not  only  by  numerical  results  but  also  by  related  theoretical 
considerations  which  we  shall  not  go  into  here. 

Better  success  was  achieved  by  following  the  alternate  schema  suggested 
in  the  material  which  follows  Theorem  2.3  in  Section  2.  We  used  a  fixed  cost 


functional  of  the  form 


(3.39) 


£  /J  (”  (R(t)2  -  a2)2  +  (u(t))2]dt 

where 

R(t)2  -  (x(t))2  +  ( x( t ) ) 2 

and  a  is  the  average  amplitude  of  the  uncontrolled  solution  when  0  -  .4. 

The  condition  W(x(t))  =  0  (cf.  (2.55)  pp)  is  not  quite  satisfied  in  this 

case  but  the  uncontrolled  orbit  is  so  nearly  circular  that  the  discrepancies 

have  negligible  effect.  When  9  *  .4  there  is  no  noticeable  modifications  of 

the  orbit  but,  as  9  is  increased  the  effect  of  the  optimal  control  policy  is 

more  and  more  pronounced,  producing  the  modified  amplitude  curve  A  (9)  shown 

c 

by  the  dashed  curve  in  Figure  3.2  and,  for  9  =  .6,  the  modified  orbit  shown 
as  a  dashed,  closed  curve  in  Figure  3.1. 

The  extreme  "fragility"  of  the  Newton  method  became  apparent  in  these 
computations!  whether  the  parameter  was  0  in  (3.38)  or  9  in  (3.37)  in 
connection  with  (3.39),  it  was  necessary  to  change  it  in  extremely  small 
increments  to  avoid  numerical  "blow-up"  -  primarily  due  to  the  instability  of 
the  adjoint  system  as  one  integrates  in  the  positive  t  direction. 

In  future  work  we  hope  to  use  the  steepest  descent  method  outlined  here 
to  compute  optimal  controls  and  modified  orbits  for  a  system  representative  of 
the  wing  -  air  system  described  in  Section  1  and  shown  in  Figure  1.2.  The 
difficulties  already  noted  for  (3.38)  indicate  that  the  use  of  the  Newton 
method  would  be  almost  hopeless  in  this  case. 
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Figure  3.2  Controlled  and 
Uncontrolled  Amplitude  vs  Parameter 
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